d0g3Alert

I have been using security onion for a long time now. I started dabbling with it while I was studying for my B.S. and have continued to use it throughout my career. It has been my go to platform for anything NSM related or a platform for learning SIEM administration. Security onion is a great platform, out of the box you’re pretty much setup to start monitoring your network as it comes with great tools that require little configuration to get started. Suricata or Snort for IDS and ZEEK or Suricata for network metadata. It even offers PCAPs along with Syslog ingestion and endpoint monitoring options. The list could really continue on.

After spending a lot of time setting up Security Onion (SO), playing with configurations then having to re-setup SO because I made a catastrophically bad config, I started to get fairly comfortable with the basics of the platform. I wanted to start really customizing it for my own purposes, so (not SO) I learned about ElastAlert (EA). ElastAlert is an alert framework for Elasticsearch that allows you to monitor for just about anything you would want¹. Out of the box you can alert for anomalies, spikes, and other patterns. It’s pretty simple to figure out how it works. Alerts are built using YAML and they run on a schedule to search for the conditions specified in the alert. Once a doc is found that matches those conditions, an alert(or doc) is written to an EA index in Elastic. In addition to just writing the alert to an index, you can specify an Alert type. The alert types give extra customization options for handling the alerts that are generated, some examples are: email, running a command, sending a Slack or Discord and the list goes on².

I wanted to get the hang of using EA with its built-in capabilities before I started going ham on customizing everything (and inevitably breaking SO having to start fresh AGAIN). I created a few basic alerts to send to my Discord server. I started out with notifications for High priority Suricata alerts, this was an easy ANY rule (Screenshot above). Then moved on to a New Term rule for any new IDS alerts that generated for the first time within a 90 day window. Since both of those were successful I started to play with a few others.

These alerts were starting to workout really well for what I wanted them for. I was getting notified for interesting IDS events along with other simple items I wanted to keep tabs on. The next thing I wanted to do was have an easy way to search for them with SO’s Hunt or Kibana. In SO you can add EA’s index to the indices that are queried through Hunt and Kibana which sort of worked ³. It didn’t really capture what I was looking for since the doc that was generated didn’t use the same fields as the other docs. So this is where I had the idea for D0g3Alert. I wanted to write an alert module for EA that would capture the fields that I wanted from the docs that were being alerted on. This meant that they would all live within their own index and be searchable in the Alerts section of Hunt.

  payload = {
    "rule": {
      "name": self.rule['rule.title'],
      "uuid": self.rule['rule.id'],
      "category": self.rule['rule.category']
      },
    "event":{
      "dataset": event_dataset,
      "module": event_module,
      "severity": self.rule['rule.severity'],
      "severity_label": rule_level,
      "timestamp": timestamp
      }
  }
  # set ES url
  url = f"https://{self.rule['es_host']}:{self.rule['es_port']}/so-playbook-alerts-{today}/_doc/"
  # post to ES index
  requests.post(url, data=json.dumps(payload), headers=headers, verify=False, auth=creds)

Writing to ES index

I took the idea from the Playbook alert module that came with SO since it does a bit of what I was looking for. The main thing that needed to be modified was how the matched doc fields were placed in the new alert doc. Reading over the EA docs, I learned that alert(self,match) comes with a list of dictionary objects that contain the info about the match. So the easy option could be to just add the match dictionary directly to the alert doc. That would work fine, but I didn’t want to have unnecessary info in the alert doc, just the important elements for that alert. I needed a way to specify the fields I wanted and then only write those to the alert doc. Well self.rule contains a dictionary with all the rule configuration options. Since those can be pulled from the yaml, I just added a list of fields to pull from the matching doc called event.fields.

event.fields:
  - '@timestamp'
  - destination.ip
  - destination.port
  - source.ip
  - network.community_id

event.fields gets pulled into createDictionary and appended to the payload

if 'event.fields' in self.rule.keys():
  payload.update(createDictionary(match, self.rule['event.fields']))

Now that I’m grabbing the fields from the matching doc, I wanted to specify some other items that would be important for tracking and correlating the alerts. Some of these fields are required for the alert to work, but others are optional and have defaults built into the alert.

• Required Fields

  • rule.title

    The name of the rule.

  • rule.id

    ID for the rule. Can be used for rule tracking and tuning.

  • rule.category

    Specify a category for the rule.

  • rule.severity

    Specify the Severity as an integer of 1-4. It will add an additional severity_level field. The severities will translate as follows; 1:low, 2:medium, 3:high, 4:critical.

• Optional Fields
  • event.dataset : Defaults to alert if omitted from rule
  • event.module : Defaults to d0g3alert if omitted from rule
  • event.fields : Specify fields that you want to be pulled from the matching doc and written to the new alert doc. Defaults to all fields from matching doc if omitted from rule
  • mitre.id : Specify the Mitre technique ID associated with this rule. If omitted, the field will not populate. Maps to rule.mitre.id.
  • mitre.name : Specify the Mitre technique name associated with this rule. If omitted, the field will not populate. Maps to rule.mitre.technique.
  • link.filters : Creates a new field called hunt_link and generates a link based on the fields provided. If omitted, the field will not populate.

Once the required fields are added you can build the rule to look for whatever it is you are hoping to highlight.

I’ve added a sample rule to my github, but feel free to pull this down and start using it within your own environment. If you can think of improvements, please feel free to submit a PR. d0g3Alert Github